Enter key image into the system; 




r 


Set training parameters and click the 
training button to teach the system 
what to look for 




f 


Enter search-directory(s) 




f 


Set search parameter(s), and click the 
search button 







The output is a web page, which is a list 
of names and weights: 

• The weight of an image is 
related to the characteristics you 
are looking for (the weight is 
similar to an Litemet search 
engine weight); 

• Click the name of each image 
and an image will pop up on the 
screen. 



Figure 1. Search Process, which is appiicabie to image verification, 
identification, and retrievai. 
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Application Layer 



Presentation Layer 



APN 



ABM 



Figure 2. 3-Layer Architecture for software implantation of tlie Present 
Invention. 
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Figure 3. Sample User Interface of the Present invention. 
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Figure 4. Sample Key Input for the Present Invention. 
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D.\imageHb\fbi\GODWIISriA.JP,G 24192 / 
D:\imaaeHb\fei\G.QDW]N2A:JP(^BMed .; 

D-:\imagdib\fbi\G6pWMA:JP.G;3315d ^^ 

D:WeHb%ftG€ifei^.(g:3ga'91^2^8 

D:\imag^iib>f6ftG^B?m^ 



Images 



Scores 



Figure 5. Sample Search Output of the Present Invention. The search 
output is a list of pairs, or doublets. 

For example, line one of the figure 5 shows a double (image, score), where the image is "D:\... 
G0DWIN1A.JPG", and the score is 24192. 
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C:\bouLhtm - Microsoft Internet EKplorer IHIil E 
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Figure 6. Sample Classification output of tlie Present Invention. The 
classification output is a list of triplets. 

The N:N classification is implemented by a set of 1:N searches. Let a and b be two different classes. 
Given an image, x, we can perform a 1:N search for class a, with a triplet (x, a, score-x-a); and a 1:N 
search for class b, with a triplet (x, b, score-x-b). The final answer is (x, score-x-a, score-x-b; max {score- 
x-a, score-x-b) ). In the above figure, the first line, "classl_9.jpg" is an image, x. The second line has 9 
scores, indicating how likely the image x belongs to any of the 9 classes. Finally, Class 0 is the best match 
because it has the maximum score, 193784. The training image for class 0 is "D:/. . ./classl jpg". The third 
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line in the above figure shows the classification result of the image, "classl_9.jpg", which is also a link to 
the training image, is "D:/. . ./class 1 jpg". 
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Enter key image into the system. 


< i 




f 


Set training parameters and click the 
training button to teach the system 
what to look for. 






f 


Enter search-direct6ry(s). 






f 



Set search parameter(s), and click the 
search button. 



No 

► 



Start classification and present results: 

• An image link for each image in 
the search directory; 

• The classification weights of 
this image in each search; and 

• The classification of this image 
as a link. 



Figure 7. Classification Process, which consists of multiple search 
processes in Figure 1. 
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Provide the batch code to the system. 




f 


Click the batch button to execute the 


code. 





Figure 8. Batch Process, which allows users to duplicate a Search 
Classifications in two clicks. 
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Combine an image and its classification into a vector. 




r 


All such together form a mathematical configuration space. Each 
point in such a space is called a state. 




f 


A Markov chain exists in such a space where the state of the 
configuration space is a state of the Markov chain. 




f 


The construction of such a Markov chain is by a particular type 
of neural network, called ABM network or APN network. 




f 



The Markov chain will settle on its invariant distribution. A 
distribution function is deployed to describe such a 
distribution. In particular, such a distribution function 
classifies the images. 



Figure 9. ABM and APN Algorithm Fiow Chart. 
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Figure 10. An example of a fully connected artificial neural network with 4 
neurons {3, 2, 1, 0}. 
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Figure 11. The Markov chain generated by the neural net with 4 neurons in 
Figure 10. 

The controlling factor in a Markov chain is the transition-probability Matrix. Here p y is. the transition 
probability matrix element from state i to state j . For example, p 12 is the transition probability from 
neuron state 0001, where the neuron 0 is excited and the rest are grounded, to neuron state 0010, where 
the neuron 1 is excited and the rest are grounded. Each neural state is a state in the Markov chain. Assume 
the neurons are {3, 2, 1, 0}. The states are 0000 = 0; 0001 = 1; 0010 = 2; 1111 = 15, i.e. the Markov 
chain state 0 is a neuron state where all neurons are grounded; the Markov chain state 1 is a neuron state 
where all neurons are grounded except neuron 0; the Markov chain state 2 is a neuron state where all 
neurons are grounded except neuron 1; the Markov chain state 3 is a neuron state where neurons 3 and 4 
are grounded and neurons 0 and 1 are excited; . . . 
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Create an ABM net with no connections 



Combine an image and its classification into an input vector. 



Impose the input vector to the leaming module. 



The ABM neural connections are calculated based on the input 
vector. Let N be the number of neurons; the order of 
connections can be up to N and the number of connections can 
be 2**N, where ** represents the exponential function. 



The Markov chain is formed after the connections are 
established. This Markov chain will settle on its invariant 
distribution. A distribution function is deployed to describe 
such a distribution. 



This distribution function, once obtained, can be used to 
classify images. This will produce triplets of image, class, and 
weight. Image retrieval and classification are two different 
sides of the same token. 



Output results: triplets of image, class, and weight. 



Figure 12. More Detailed ABIVI Algorithm Flow Chart. 
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Create an ABM net with no connections. 



± 



Combine an image and its classification into an input vector. 



Impose the input vector to the learning module. 



The APN neural connections are calculated based on the input vector. 
Let N be the number of neurons; the order of connections can be up to 
N and the number of connections can be 2**N, where ** represents the 
exponential function. 



A mapping over each connection is estabUshed. 



The K-element mapping is changed to N-element mapping. By taking 
the domain of this mapping away, the range of this mapping forms a 
vector, APN connection vector. 



The Markov chain is formed after the connections are established. This 
chain will settle on its on its invariant distribution. 



This distribution function, once obtained, can be used classify images. 



Comparing the input-vector and the APN-connection-vector modifies 
this weight. This will produce a new set of triplets of image, 
classification, and weight. 



Output results: triplets of image, class, and weight. 



Figure 13. More Detailed APN Aigorithm Flow Chart. 

The APN algorithm is an extension of the ABM algorithm. The ABM algorithm is for binary images and 
the APN algorithm is for multi-valued images. The APN algorithm will first use the ABM algorithm to 
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make an image matching for binary images. If there is no match from the ABM algorithm, the ABM 
algorithm will output a score 0, which will remain to be 0 when the APN algorithm finishes. If there is a 
match for the ABM algorithm, then the APN will further modify the ABM results by comparing two 
input vectors generated by the two comparing images. For example, consider two images (3 5 0 0) and (1 
2 0 0), their binary versions are 1100 and 1100. The ABM algorithm will compare the binary images, 
001 1 and 1 100, and return a match. The APN algorithm will take the ABM results, and modify it by the 
distance between (3 5) and (1 2). 
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(a) Three-dimensional connection space. 




(0000) Xi (1000) 



(b) Four-dimensional connection space. Assume two connections are 1100 and 1111, then the 
Sensitivity distance between them is 2, if the Li distance is used, where Li = | xa - xi | + | y2 - 
yi I + ... Here 1100 means that neuron 0 and neuron 1 have a non-zero connection matrix 
element; and 1111 means there exists a non-zero matrix element between neuron 0, 1, 2, and 3. 

Figure 14. Connection Space and the Sensitivity distance. 
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(a) Three-dimensional binary image space. 




(0000) Xi om 



(b) Four-dimensional binary image space. Assume a binary image is 1111, we can draw a 
"sphere" around 1111 with a radius of 1. The following binary images are in this sphere: 1111, 
0111, 1011, 1101, 1110. All these images in this "sphere" will be considered the same as the 
original image, 1111. Blurring is a user-defined parameter that defines this radius. Any image in 
this sphere is as good as the original image, 1 1 1 1 , for image matching. 

Figure 15. Image Space and the Blurring distance. 
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Create an ABM net with no connections. 




f 


Combine an image and its classification into an input vector. 




f 



The ABM neural connection matrix is initiated based on the 
input vector. The image is randomly broken down into a 
predefined number of pieces. 



1 


r 


Get the configuration vector for each image piece. 




f 


Make configuration- vector = connection-vector, which results 
in the first synaptic connection matrix element. 




f 


Calculate the rest of the matrix elements via the first matrix 
element and the distances in the connection space. 




f 


The Markov chain is formed after the connections are 
established. 



Figure 16. (a) ABM Algorithm. 
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i 


For each connection, add an additional mapping. 



Figure 16. (b) APN algorithm, which is an extension of the ABM algorithm for non-binary values. For 
example, consider two images (3 5 0 0) and (1 2 0 0), their binary versions are 1 100 and 1 100. The ABM 
algorithm will compare the binary images, 0011 and 1100, and retum a match. The APN training 
algorithm will first train the binary ABM net, and then store an additional mapping (O-* 3, 1^ 5) for 
each connection. 

Figure 16. ABM and APN Learning Algorithm Flow Chart. 
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An image to be classified is imposed on the Markov Chain. 



This Markov chain will settle on its invariant distribution. A 
distribution function is deployed to describe such a distribution. 



This distribution function, once obtained, can be used to classify 
images. This will produce triplets of image, class, and weight. 



These triplets of image, classification, and weight can be viewed 
as the result of the classification process. 



Figure 17. ABM Recognition Algorithm Fiow Chart. 
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An image to be classified is imposed on the Markov Chain. 




f 


This Markov chain will settle on its invariant distribution. A 
distribution function is deployed to describe such a distribution. 






This distribution function, once obtained, can be used to classify 
images. This will produce triplets of image, class, and weight. 




r 



Let s be the new APN connection vector generated by an input 
search image. Let t be the APN connection vector generated by 
the training vector. A distance between s and t, either LI 
distance or L2 distance, can be calculated. The new weight will 
be directly proportional to the old weight and inversely 
proportional to this distance. The present invention covers all 
functions of obtaining the new weight: 
New weight = f (old weight, distance). 
This will produce a new set of triplets of image, classification, 
and weight. 



These triplets of image, classification, and weight can be viewed 
as the result of the classification process. 



Figure 18. APN Recognition Algorithm Flow Chart. 
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